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1 Problem statement 



This paper is concerned with the numerical solution of systems of coupled monotone inclusions in 
Hilbert spaces. A simple instance of this problem is to 

find x\ G Tt, x 2 & "H such that J e 1 1 + 1 2 (1-1) 

1^0 G A 2 x 2 + x 2 - xi, 

where (TC, \\ ■ ||) is a real Hilbert space, and where Ai and A 2 are maximal monotone operators acting 
on TC. This formulation arises in various areas of nonlinear analysis |20| . For example, if A\ = df\ 
and A 2 = df 2 are the subdifferentials of proper lower semicontinuous convex functions /1 and f 2 
from TL to ]— 00, +00], is equivalent to 

minimize fi(xi) + f 2 (x 2 ) + \\\xi - x 2 \\ 2 . (1.2) 

XlSTt, X2&H I 

This joint minimization problem, which was first investigated in [1], models problems in disciplines 
such as the cognitive sciences [9], image processing [34J, and signal processing [38] (see also the 
references therein for further applications in mechanics, filter design, and dynamical games). In 
particular, if f% and f 2 are the indicator functions of closed convex subsets C\ and C 2 of 7i, (11.21) 
reverts to the classical best approximation pair problem [191 [28j [4"0] 

minimize H^i — x 2 \\. (1-3) 
On the numerical side, a simple algorithm is available to solve (jl.ip . namely, 

xi, G H and (Vn G N) < (1.4) 

[Xl,n+1 = (Id ^2,n- 

This alternating resolvent method produces sequences (xi )n ) nS N and (x2, n )neN that converge weakly 
to points x\ and X2, respectively, such that (xi,^) solves (jl.ip if solutions exist [201 Theorem 3.3]. 
In [5], the variational formulation (II. 2D was extended to 

minimize /i(aci) + /2(a; 2 ) + oll-^i^l - L 2 x 2 \\g, (1.5) 

xig7-(i,a;2Gn2 ^ 

where 7ii, 7^2, and are Hilbert spaces, f\ : Hi — » ]— 00, +00] and / 2 : W2 — > ] — °o> +00] are proper 
lower semicontinuous convex functions, and L\ : H\ — > and L2 : W2 — * Q are linear and bounded. 
This problem was solved in [5] via an inertial alternating minimization procedure first proposed in 
[9] for the case of the strongly coupled problem (jl.2p . 

The above problems and their solution algorithms are limited to two variables which, in addition, 
must be linearly coupled. These are serious restrictions since models featuring more than two 
variables and/or nonlinear coupling schemes arise naturally in applications. The purpose of this 
paper is to address simultaneously these restrictions by proposing a parallel algorithm for solving 
systems of monotone inclusions involving an arbitrary number of variables and nonlinear coupling. 
The breadth and flexibility of this framework will be illustrated through applications in the areas of 
evolution inclusions, dynamical games, signal recovery, image decomposition, best approximation, 
network flows, and decomposition methods in Sobolev spaces. 

We now state our problem formulation and our standing assumptions. 
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Problem 1.1 Let (Hi) i<i<m be real Hilbert spaces, where m > 2. For every i E {1, . . . ,tti}, let 
Aj : Hi — ► 2 Wi be maximal monotone and let B{ : Hi x • • • x H m — * H%. It is assumed that there 
exists j3 6 ]0, +oo[ such that 

(V(an,...,x m ) e Wi x ••• x H m )(V(yi,...,y m ) € Hi x • • • x W m ) 

m m 

^ . . . ,x m ) - . . .,y m ) \x i -y i )>f3 s ^ ||-Bi(a;i, • • -,x m ) - Bi(y 1} . . . ,y m )\\ . (1.6) 

i=l i=\ 

The problem is to 



find x\ E Hi, . . . , x m E W m such that < 



under the assumption that such points exist. 



[,..., j 



E Aixi + Bx(xi, 
E A m x m -\- -Bm (xi , . . . , x m ) , 



(1.7) 



In abstract terms, the system of inclusions in (jl.7p models an equilibrium involving m variables 
in different Hilbert spaces. The ith inclusion in this system is a perturbation of the basic inclusion 
E AiXi by addition of the coupling term Bi(x\, . . . , x m ). This type of coupling will be referred to 
as weak in that it is not restricted to a simple linear combination of the variables as in (|l.lj) . As 
will be seen in Section 01 our analysis captures various linear and nonlinear coupling schemes. For 
example, if 

(Vi E {1, . . . , m}) Hi = H and (Vx E H) B { (x, . . . ,x) = 0, (1.8) 

then Problem 11.11 is a relaxation of the standard problem [30|, ITT] of finding a common zero of 
the operators (^4j)i<j< m , i.e., of solving the inclusion E Di=i ^ n particular, if m = 2, 

Hi = = H, B\ = —B2: (xi,X2) 1— » xi — X2, and /? = 1/2, then Problem 11.11 reverts to 
(jl.ip . On the other hand, if m = 2, yli = A2 = 8/2, B\ \ (x\,x%) 1— * L\(L\x\ — L2X2), 
B 2 : (xi,x 2 ) i-» -L^iixi - L 2 x 2 ), and /3 = (||ii|| 2 + ||L 2 || 2 ) -1 , then Problem [LT1 reverts to (|1.5p . 



The paper is organized as follows. In Section [21 we present our algorithm for solving Problem ll.il 
and prove its convergence to solutions to Problem 11.11 In Section [3l we describe various instances 
of (II. 7p resulting from specific choices for the operators (Aj)i<j< m , e.g., minimization problems, 
variational inequalities, saddle-point problems, and evolution inclusions. In Section [H we discuss 
examples of linear and nonlinear coupling schemes that can be obtained through specific choices 
of the operators (-E>i)i<i< m in Problem 11.11 Applications to systems of evolution inclusions are 
treated in Section [5l Section [6] is devoted to variational formulations deriving from Problem 11.11 and 
features various special cases. The applications treated in that section include dynamical games, 
signal recovery, image decomposition, best approximation, and network flows. Finally, Section [7] 
describes an application to decomposition methods in Sobolev spaces. 

Notation. Throughout, H and (Hi)i<i< m are rea l Hilbert spaces. Their scalar products are 
denoted by (• | •) and the associated norms by || • ||. Moreover, Id denotes the identity operator on 
these spaces. The indicator function of a subset C of H is 

/O, if x E C; 
i C -x^ < (1.9) 
I +00, if x f. C 
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and the distance from x G TL to C is dc{x) = inf yg c 11^ ~~ 2/||> if C is nonempty closed and convex, 
the projection of x onto C is the unique point Pqx in C such that ||x — Pcx\\ = dc{x). We denote 
by ro(7^) the class of lower semicontinuous convex functions / ': TL — > ]— oo,+oo] which are proper 
in the sense that dom/ = {x G W | /(x) < +00} 7^ 0. The subdifferential of / € ro(7i) is the 
maximal monotone operator 

a/:W^2 H :x^{neH| (Vy G W) (y - x | u) + /(a:) < f(y)}. (1.10) 

If ^ is a real Hilbert space, B (TL,G) is the space of bounded linear operators from TL to Q and 
B (W) = B (TL,TL). We denote by graA = { (x, u) G H x H | u£ Ax} the graph of a set-valued 
operator ^4: TL —>■ 2 n , by domi = [x e TL | ylx 7^ 0} its domain, and by Ja = (Id +j4) _1 its 
resolvent. If A is monotone, then J a is single- valued and nonexpansive and, furthermore, if A is 
maximal monotone, then dom J a = TL. For complements and further background on convex analysis 
and monotone operator theory, see [HI [25J E21 165| [66] . 



2 Algorithm 

Let us start with a characterization of the solutions to Problem II. li 

Proposition 2.1 Let (xj)i<j< m G Tt\ X • • • x Zei (Aj)i<j< m G [0, l[ m , and /ei 7 G ]0,+oo[. 
Then (xj)i<j< m solves Problem \l.l\ if and only if 

(Mi G {1, . . . ,m}) Xi = X i x i + (l-X i )J 7 A i [xi-jB i (xi,...,x m )). (2.1) 

Proof. Let i G {1, . . . ,m}. Then, since B{ is single- valued, 

G AiXi + Bi(xi, . . . ,x m ) 44> Xi-jBi(xi,...,x m )exi + ^AiXi 

44> Xi = J 7 Ai(xi -lBi(xt,... ,x m )) 

o Xi = Xi + (1 - \i){J lA .{xi-^Bi(xi, . . . ,x m )) - Xi), (2.2) 

and we obtain (12.11). □ 



The above characterization suggests the following algorithm, which constructs m sequences 
((^i,n)neN)i<i<m- Recall that is the constant appearing in (jl.6j) . 



Algorithm 2.2 Fix e G ]0, min{l, /?}[, (7„)„ e N in [e,2(3 - e], (A n ) neN in [0, 1 - e], and (xj )0 )i<i< m 
G Hi x • • • x Tim- Set, for every n G N, 

£l,n+l = Ai + (1 - Ai, n ) (j ln A 1>n {xi,n ~ J n (Bl,n(xi + ai 

m,n+l — X m ^ n X m ^ n -\- (1 A mjn ) ^>^7 nJ 4 m , n (xm,n 7n(-^m,ra 0^1,71) • • • > ^m.n) "I - ^m,n)) ~t~ Qm.n^ j 

^ (2.3) 

where, for every i G {1, . . . , m}, the following hold. 

(i) (^4i,n)neN are maximal monotone operators from TLi to 2 Wi such that 

(VpG]0,+oo[) SU P ll J 7nA iin y - JynAM < +00. (2.4) 

n 6 N Hfll^ 
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(ii) (-Bj, n )neN are operators from Hi x • • • x H m to Hi such that 

(a) the operators {B^ n — Bi) n( zn are Lipschitz-continuous with respective constants (fn in ) ne N 
in ]0, +oo[ satisfying Yl n &i Ki > n < +°°' anc ^ 

(b) there exists z G Hi x • • • x H m , independent of i, such that (Vn G N) Bj )TI .;s = -BiZ. 

(hi) (ai, n ) n6 N and (&i, n )neN are sequences in Hi such that J2n€N ll a *,«ll < +°° and Ylnm \\Kn\\ < 

+OO. 

(iv) (A ijn ) ne N is a sequence in [0, 1[ such that Yln&N Wn ~ K\ < 



Conditions (i) and (ii) describe the types of approximations to the original operators (Ai)±< 



i<m 



and (-£>i)i<i< m which can be utilized. Examples of approximations will be provided in Proposi- 



tion 13.71 and Remark 14.71 respectively. Condition (hi) quantifies the tolerance which is allowed in 



the implementation of these approximations (see [33 ^1421145] for specific examples), while (iv) quan- 
tifies that allowed in the agent-dependent departure from the global relaxation scheme. The parallel 
nature of Algorithm 12.21 stems from the fact that the m evaluations of the resolvent operators in 
(|2.3p can be performed independently and, therefore, simultaneously on concurrent processors. 

Our asymptotic analysis of Algorithm 12.21 requires the following result on the convergence of the 
forward-backward algorithm. This algorithm finds its roots in the projected gradient method [38] 
and certain methods for solving variational inequalities [151 [26| [4"9l [6T] (see also the bibliography of 
[31] for more recent developments). First, we need to define the notion of cocoercivity. 



Definition 2.3 Let x £ ]0, +oo[. An operator B : H —* H is x- cocoerc i ve if 

(Vx G H)(\/y G H) {x - y \ Bx - By) > X \\Bx - Byf . (2.5) 
If X = 1 in (|2.5p . then B is firmly nonexpansive. 

Lemma 2.4 [31, Corollary 6.5] Let (Ti, \\\ ■ |||) be a real Hilbert space, let x £ ]0, +oo[, let A: 7i — > 
2^ be a maximal monotone operator, and let B : 7i — > H be a X' cocoerc 'l ve operator such that 
{A + B)~ 1 {Q) 7^ 0. Fix e G ]0, min{l, x}[ , let (jn)neN be a sequence in [e, 2% — e], let (A n ) ng N be a 
sequence in [0, 1 — e], and let (a n ) ng N and (6 n ) n eN be sequences in 7i such that J2neN lll a ™lll < +°° 
and X^nGN lll^™lll < +oo. Fix xq G 7i and, for every n G N, set 

X n +1 = Kx n + (1 - \n){J ln A{x n ~ ln{Bx n + b n )) + d n ) . (2.6) 

Then (x n ) ne fq converges weakly to a point in (A + £?) -1 (0). 

We shall also use the following fact. 

Lemma 2.5 [3H Lemma 2.3] Let (H, \\\ ■ \\\) be a real Hilbert space, let x G ]0, +oo[, let B: H, — > Ti 
be a x- cocoerc i ve operator, and let 7 G ]0, 2%[. Then Id — 7I? is nonexpansive. 

The main result of this section is the following theorem. 

Theorem 2.6 Let ([xi in ) n ^)i<i< m be sequences generated by Algorithm \2.3l Then, for every 
i G {l,...,m}, (xj jn ) n£ N converges weakly to a point Xi G Hi, and (xj)i<j< m is a solution to 
Problem\l.l[ 
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Proof. Throughout the proof, a generic element x in the Cartesian product TL\ x • • • x TL m will 
be expressed in terms of its components as x = (xi)i<i< m . We shall show our algorithmic setting 
reduces to the situation described in Lemma 12.41 in the real Hilbert space H obtained by endowing 
TL\ x • • • x TL m with the scalar product 



((• I •»: ( x ,y) ^ ( Xi I Vi 



1=1 



with associated norm 



: x i — ^ 



(2.7) 



(2- 



To this end, we shall show that the iterations (|2.3p can be cast in the form of (|2.6p . First, define 

m m 

A: 7i ^2 H : x ^ XAiXi and (Vn G N) A n : H -> 2 H : x \-> X A, n Xi- (2-9) 

8=1 8=1 



It follows from the maximal monotonicity of the operators (^4i)i<i< m > condition (i) in Algorithm [2721 
([ZZD , and (J2T9]) that 

A and (A n ) rag pj are maximal monotone, (2-10) 

with resolvents 

Ja: 7i — > H : x h-> (Ja^)^^™ and (Vn € N) Ja„ : H^H : x (J A . n Xi)i<i< m , (2.11) 



respectively. Moreover, for every p G ]0,+oo[, we derive from (|2.8|) . (|2.1ip . and condition (i 
Algorithm [27J that 



m 



\\\y\\\<p 



\\\<p\ 7=t 



SU P W J 1nA t , n yi - J ln AiVi\\ 

neN IIMII<p i=l 

m 

< Yl SUP 'I J ^Ai, n Vi ~ JlnAiViW 
i=l nGN llf»II^P 

< +00. 

Now define 

B: H ^ H: x (Bix)i<i< m and (Vn G N) B n : H. — ► "H: x (-»• (Bi, n aOi<i<m- 
Then (|1.7|) is equivalent to 

find x £ 7i such that G Aa: + Bs. 
Moreover, in the light of (pTTj) . (f278|) . and (p7T3j) . (fL6l) becomes 

(Va: G «)(Vy 6«) ((x - y \ Bx - By)) > (3\\\Bx - By\\\ 2 . 



(2.12) 



(2.13) 



(2.14) 



(2.15) 
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In other words, B is /3-cocoercive. Next, let n E N and set 

c n = ifli,n)l<i<m an d = (&i,n)l<i<m- 



We deduce from (|2.8j) and condition (iii) in Algorithm 12.21 that 

£iwi<E 



fceN 



* £ IKfcll 2 < £ W ai > k W < +0 ° 

\ i=l i=l fceN 



and, likewise, that 



Now set 



y~] iiidfciii < +00. 

fcGN 

a^n = (2;«,n)i<j<m and A n : ?i — >■ H : x h-> (Aj in Xj)i<j< r 



It follows from (|2.8[) and condition (iv) in Algorithm 12.21 that 
IIIA 



mil = max Aj jn < 1 and |||Id — A n ||| = 1 — min Aj„ < 1. 

l<i<m ' l<i<m 



(2.16) 

(2.17) 

(2.18) 
(2.19) 
(2.20) 



Hence, 



where 



|A n ||| + |||Id - A n ||| = 1 + max (Ai,„ - A n ) - min (A i>n - A n ) < 1 + r n , (2.21) 

KKra Ki<m 



r n = 2 max |A» n - A n |. 

Ki<m 



We observe that, by virtue of condition (iv) in Algorithm 12.2 



ET fc = 2} max \X ik - X k \ < 2} } \X ik - A fe | < +00. 



(2.22) 



(2.23) 



fceN fceN i=l fceN 

Moreover, in view of (12. lip . (I2.13p . (I2.16p . and (|2.19j) . the iterations (12. 3p are equivalent to 

aj n + (Id - A n )(J 7nAn (x n -"f n (B n x n + d n )) +c„). (2.24) 

Now define 



D n — B n — B. 



(2.25) 



It follows from condition (ii)(a) in Algorithm 12.21 (|2.8p . and (|2.13p that D n is Lipschitz continuous 
with constant K n = w Si=i K f n an< ^ that 



fceN fceN 



\ i=l 



i=l fcGN 



Furthermore, set 



t> n — D n x n -\- d n 
and let s be a solution to Problem I l.li Then 



(2.26) 
(2.27) 



|^n||| — 1 1 1 -^n^n 1 1 1 H~ |||^n||| 

— \\\-D n X n -^n^||| H - |||-^n^ -^n^lll H~ |||^n| 

< K n (\\\x n - x\\ \ + \ \\x - z\\\) + \ \\d n \\\, 



(2.28) 
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where z is provided by assumption (ii)(b) in Algorithm 12. 21 We now set 

T n = Id - ln B and e n = J lnAn (T n x) - x. (2.29) 
On the one hand, the inequality sup fceN jk < 2/3 yields 

|||T„a:||| < p, where p = \\\x\\\ + 2/?|||.Ba:|||. (2.30) 
On the other hand, Proposition 12.11 and (|2.11|) supply 

x = J lnA {T n x). (2.31) 
Therefore, (j239j) . (|230j) . and <|212|> imply that 

E III^IH = ^lll^ fc A fc (^)-^||| = Y,\W J i k A k (T k x)- J lkA {T k x)\\\ <+oo. (2.32) 



fcGN fceN 
In addition, (|2351 . (|23TD . and (f239l) yield 

J<y n An \ x n ~ r fn{^n x n + <i n )) — 33 = J<y nAn (T n X n — 7ra^n) — J-y n A n {T n x) + e n . (2.33) 

Since J 7n A is nonexpansive as a resolvent (see [1 H Proposition 3.5.3] or |25t Proposition 2. 2. hi)]) 
and T n is nonexpansive by Lemma 12.51 we derive from (|2.33p and (12.28P that 

<En ^fn{B n X n + d n )) 3j||| < 1 1 1 <Zy n An (T n X n Tn^n ) - J ln A n {T n x)\\\ + \\\e n \\\ 

^ 1 1 l^n^n Ifnbn T n x\ 1 1 ~h 1 1 \&n\ \ \ 
— III ^111 In 1 1 1 1 1 1 \ \\ &n \ \ \ 

< \\\x n - x\\\ + 2/?|||b n ||| + |||e n ||| 

< (1 + 2(3K n )\\\x n - x\\ \ + 2(3K n \\\x — z\\\ 

+ 2/?|||d n ||| + |||e n |||. (2.34) 

Thus, it results from (12341 . (12341 . (I23TI) . and (123(1 that 

|| |ac n+1 -ac||| = \ \\A n (x n - x) + (Id - A n )(J lnAn (x n - j n (B n x n + d n )) - x + c n )\\\ 

< |||A n ||| |||x n - x||| + 1 1 1 Id - A n ||| |||c n ||| 

+ 1 1 1 Id. - A n ||| \\\J lnAn (x n - j n (B n x n + d n )) - x\\\ 

< |||A n ||| |||x n - x\\ \ + 1 1 1 Id - A n ||| |||c n ||| 

+ 1 1 1 Id - A n ||| ((1 + 2(3K n )\\\x n -x\\\+ 2(5K n \\\x - z\\\ 
+ 2l3\\\d n \\\ + \\\e n \\\) 

< (|||A n ||| + 1 1 1 Id - A n |||)|||x n - x||| + |||Id - A n ||| (|||c n ||| + 2(3K n \\\x n - x\\\ 
+ 2f3K n \\\x- z\\\+2f3\\\d n \\\ + \\\e n \\\) 

< {1 + T n )\\\x n - x\\ \ + \ \\c n \\ \ + 2j3n 

n\\\ <En ^111 

+ 2fiK n \\\x - z\\\ +2/?|||d n ||| + |||e„||| 

< (l + a n )|||x n -a!||| + * n , (2.35) 



where 



a n = T n + 2(5K n and 5 n = \\\c n \\\ + 2(3K n \\\x - z\\\ + 2/3|||d n ||| + |||e n |||. (2.36) 
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In turn, it follows from (l2T23jh (|BD . pTTjl . (I21H1) . and (12T32D that ^ fceN a fc < +00 and EfceN <*fc < 
+00. Thus, ([235]) and [551 Lemma 2.2.2] yield 

sup |||£Cfc — a;||| < +00 (2.37) 

fceN 

and, using (f!T26j) and (f2~T8|) . we derive from (pl28j) that 

^IH&fclH < +00. (2.38) 

fceN 

In view of (j2T2?j) and (j2T29j) . (pT24]l is equivalent to 

;c n+ i = A n x n + (Id - A n ) (J lnA (T n x n - -y n b n ) + h n ) , (2.39) 

where 

= J-ynA n (T n x n - 7„b„) - J ln A(T n x n - j n b n ) + c n . (2.40) 

Now set n = sup fcgN 1 1 \x k — x\ \ | + p + 2/?sup fcgN |||&fc|||- Then it follows from (|2.37|) . and (|2.38p that 
[i < +00. Moreover, we deduce from the nonexpansivity of T n and (I2.30P that 

1 1 \T n -Kn 1 1 ^ 1 1 \T n -^n ^"n^l 1 1 "I - 1 1 I^Vi^-l 1 1 "I" 2/?| | \b n \ 1 1 

< |||x n -x|||+/o + 2^|||6 n ||| 

< H- (2.41) 
Hence, appealing to (|2.12|) and (|2.17p . we deduce from (|2.40p that 

\\\h k \\\ < +00. (2.42) 

fceN 

Note that, upon introducing 

a n = h n + 1 ( A n - A n Id ) [x n - J~ jnA (T n x n - j n b n ) - h n ) , (2.43) 
t — A n 

we can rewrite (I2.39P in the equivalent form (12. 6p . namely 

x n +\ = Kx n + (1 - K)(J ln A{x n ~ ln{Bx n + b n )) + a n ). (2.44) 
Using (|2.3ip and the nonexpansivity of J 7ii a and T n , we get 

1 1 1 J-ynA^T n X n ^/n^n) I — II fan •''III II I J~i n A(T n x) Jy n j^(T n X n Tn^n) 1 1 1 

+ \\\ h n\\\ 

<2|||x n -x|||+2/?|||6 n ||| + |||/» n |||. (2.45) 

Therefore, we derive from £07]), <|CT]> . and (12^421) that 

1/ = suplll^Efc - J lk A(T k x k - 7 fc 5fc) ~h k \\ \ < +00, (2.46) 
fceN 

and hence, from (|2.43p and the inequality A n < 1 — e, that 

1 1 | a n| 1 1 ^ 1 1 \h n \ 1 1 + - - 1 1 1 A n — A n Id J J I 1 1 fa n — J~f n A{T n X n — 7n^n) — h-n\ 1 1 
J- — 

<lll^n||| + - max |A i>n -A n |. (2.47) 

£ Ki<m 
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Thus, using (12.421) and arguing as in (12.23f) . we get 

^2 \\\ a k\\\ < +00. (2.48) 

fcGN 

However, LemmaEH asserts that, under (I2.10j) . (12,15p . (12.38p . (I2.48h . and the hypotheses on (7 n ) ng N 
and (X n )neN in Algorithm 12.21 the sequence (x n ) n< =fq generated by (|2,44p converges weakly to a 
solution to (12. 141) . Since (12.441) is equivalent to (f23l) . and (12.14p is equivalent to (fl~7l) . the proof is 
complete. □ 



3 Specific scenarios 

Problem ll.il covers various scenarios, depending on the type of operators (^4i)i<i< m utilized in (|1.7p . 
We now provide some specific examples which will serve as a basis for the concrete problems to be 
discussed in Sections [5H3 

Example 3.1 Suppose that, for every i G {l,...,m}, Ai = dfi where fi G ro(Wj). Then (|1.7|) 
reduces to the system of coupled variational inequalities 

find x\ G Hi, . . . , x m G Tt m such that 

(V*€{l ) ...,m})(V J /€W i ) {x i -y\B i (x 1 ,...,x m )) + f t (x i )<f i (y). (3.1) 

A particular case of this type of problem will be investigated in Section [6l 

Example 3.2 Suppose that, for every i G {1, . . . , m}, Ai is the normal cone operator to a nonempty 
closed convex subset C{ of Tti, that is 

A = N Ci : W, -> 2* : x ~ { G H ' SUP ^ {lJ ~ X 1 U) ~ °>' if X G Q; (3.2) 

I 0, otherwise. 

Then (|1.7p becomes a system of coupled variational inequalities of the form 

find X\ G Ci, . . . , x m G C m such that 

(Vie{l,...,m})(VyeC i ) ( Xi - y \ Bi( Xl , . . . ,x m )) < 0. (3.3) 



Such formulations will be investigated in Example 16.61 and Example 16.91 

Example 3.3 For every i G {1, . . . , m}, let 3^ and Zi are real Hilbert spaces, and suppose that 

A i -y i @Z i ^ 2*'®^: (y,z) ^ {(«,*;) G # © ^ | « G d(-Fi(; z))(y) and « G 0(i?i(y, .))(«)}, 

(3.4) 

where Fj : 3^ © — ► [— 00, +00] satisfies 

(i) (3(y , 20) 6^9 ^i)(V(y, «) G # © 2<) Fi(y , z) > -00 and F;(y, z ) < +cx>; 

(ii) for every (y, z) G 3^ © -Zi, — -z) and i*i(y, •) are lower semicontinuous and convex. 
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Then, for every i G {1, . . . , m}, Ai is a maximal monotone operator acting on Hi = y i @Z{ [57] and, 
upon setting B{ = (Bn,Bi2), where Bn : Hi — ► 3^ and ffo : Hi ^ Zi, (|1.7|) reduces to the system 
of coupled saddle-point problems 

find xi = G Hi, x m = (y m , z m ) G H m such that (Vi G {1, . . . ,m}) 



sup Fi(y,Zi) - (y | . . . ,x m ))-y. = F^y^Zi) - {yi \ B a (xi, . . .,x m )) y . 

inf Fi(yi,z) + (2 | B i2 (xi, . . .,x m )) z , = Fi(y u zi) + | B i2 (x 1 , . . .,x m )) z .. 

Such formulations will arise in Example 14.111 



(3.5) 



Example 3.4 Let us recall some standard notation [2SJ[DU]. Fix T G ]0, +oo[ and p G [l,+oo[. 
Then D(]0, T[) is the set of infinitely differentiable functions from ]0, T[ to R with compact support 
in ]0, T[. Given a real Hilbert space H, C([0, T]; H) is the space of continuous functions from [0, T] to 
H and L p ([0, T]; H) is the space of classes of equivalences of Borel measurable functions x: [0, T] — > 
H such that J \\x(t)\\ p H dt < +00. L 2 ([0,T];H) is a Hilbert space with scalar product (x, y) 1— > 
Jq (x(t) I y(t)) H dt. Now take x and y in L 1 ([0,T];H). Then y is the weak derivative of x if 

/ (j)(t)y(t)dt = -/ (d(j)(t)/dt)x(t)dt for every G D(]0,T[), in which case we use the notation 
y = x' . Moreover, 

W 1 ' 2 ([0,r];H) = {xGL 2 ([0,T];H) | x G L 2 ([0,T]; H)}. (3.6) 

Now, for every i G {1, . . . , m}, let Hj be a real Hilbert space, let Aj : Hj — ► 2 H > be maximal monotone, 
let Bj : Hi x • • • x H m — > Hj, and set Hi = L 2 ([0, T]; Hj). Then, under standard assumptions, the 
operator 

Ai - H .^2 H >:x^ l x ' + AiX > if x G H /1,2 ([0, T]; Hj) and x(0) = x(T); ^ 
[0, otherwise 

is maximal monotone (see [ID], [251 Section 3.6], [62]). In this context, with a suitable construction 
of the operators (-Bj)i<i< m in terms of (Bj)i<j< m , (II. 7h assumes the form of the system of coupled 
evolution inclusions 

findxi G W 1 ' 2 ([0,T};H 1 ),...,x m G W 1>2 ([0,T]; H m ) such that 

f0 6x , i (t) + A<(ar i (t)) + B i (xi(t),... ) x m (t)) a.e. on ]0,T{ 
{l 1 ""' m|J \x i (0)=x i (T). l ' j 

This type of problem will be revisited in Section [5j 



In Algorithm 12.21 maximal monotone approximations (^4i.n)i<i< m to the original operators 
(Ai)i<i< m can be used at iteration n, as long as (|2.4j) is satisfied. In order to illustrate this condition, 
we need a couple of definitions and some technical facts. 

Definition 3.5 Let A and B be set-valued operators from H to 2™ and let g G ]0,+oo[ be such 
that £^n(graAUgra£) ^ 0, where 22 e = {(x,y) £ H x H \ max{||x||, \\y\\} < g}. The ^-Hausdorff 
distance between A and B is Section 1.1] 

haus e (A, B) = max I sup dgravlCz), sup d gra , B (z) \ . (3.9) 

L zeEgllgraB zeE e ngra,A J 
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Moreover, the Yosida approximation of A of index 7 G ]0, +oo[ is "M. = (Id — J 7 a)/7 |114 I25|. 

Lemma 3.6 Let A: TL — » 2 W 6e maximal monotone, let x £ Tt, let j <E ]0, +oo[, and /ei /i G ]0, +00 [. 
T/ien £/ie following hold. 

(i) ^1 = J 7 a(i + (1-7/p)(^-3;)). 

(ii) 7 < ^ =^ || J 7 ax — x|| < 2|| J m ax — x\\. 

(iii) J 7 (^) x = x + 7(^(7+^2; - x)/(7 + fJ,). 

(iv) || J ' - J 7 ax|| < 2/x|| J 7 ax - x\\/(j + n). 



Proof, (i): See [251 Section II.4]. 



(ii) Set A = 7//x and observe that A G ]0, 1]. It follows from the nonexpansivity of J 7 a 
[TT| Proposition 3.5.3] and (i) that ||J 7 ax — x|| < ||J 7 ax — </^ax|| + ||^ax — x\\ = ||J 7 ax — 
J 7 A (Ax + (1 - A) J m ax) || + || J m ax - x|| < ||x - Ax - (1 - A) J m ax|| + || J m ax - x|| < 2|| J^ A x - x\\. 



(iii) This identity follows at once from the semigroup property 7+/i A = 7 (^A), which can be 



found in |25t Proposition 2.6(h)]. 



(iv) : It follows from (iii) that 
lr 7 (' 1 A)*^ ^7Ax|| 



X + 



7 



7 + H 



(j(j+fj,)A x — X^ — J 7 AX 



7 



< 



7 + ^ 

7 



(j^+^AX - x - (J 7 ax - x)j - ^ ( J 7 ax - x) 



,I^(7+m)ax — ^axU H — — II J 7 ax — x| 

7 + /U v,p; 7 + fj, 



(3.10) 



On the other hand, it follows from (i) and the nonexpansivity of </( 7 +^)a that 



\J(f+H)A x <^7AX| 



<^\\J,AX-X\ 

7 



(x+ (] 



7 + /^ 

7 



( J 7 ax - a;)) 



(3.11) 



which, combined with (|3.10p . yields the announced inequality. □ 

Proposition 3.7 Let i G {1, . . . , m} and let (7 n )neN be as in Algorithm \2.$h Then condition (12.41) 
holds if one of the following is satisfied for every n G N. 

(i) Ai in = (ai,nhn)Ai, where (7i, n )neN lies in ]0,2/3[ and satisfies J2 n eN l7i,n ~ 7n| < +00. 

(ii) A i>n = ^- n Ai, where {m,n)nes lies in ]0,+oo[ and satisfies YlneN^hn < +°°- 

(iii) 7„ = 7 G [e, 2/3 - e], and 

(V# G [ || J 7 AiO|| max{l, I/7}, +00 [) ^ haus e (A, A,n) < +00. (3.12) 
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Proof. Let p G ]0, +00 [. Since sup ngN 7„ < 2/3, we derive from Lemma E^ii)J and the nonexpansivity 
of Id - J 2j 3A i = J(2pAi)-i tllat 

(Vn G N)(Vy G Hi) \\J lnAl V - v\\ < 2\\J 2pAi y - y\\ 

< 2||(Id -J 2 p Ai )y - (H -J2(3aM + 2\\J 2P aM 



<2\\y\\+2\\J 2/3Ai 0\ 



(3.13) 



In addition, set \i = 2p + 2|| J2 / gA i 0||. We now prove assertions (i) - (iif 



(i) : It follows from Lemma l3,(^(i)| and the nonexpansivity of J 7i nAi that 



(Vn G N)(Vy G Hi) || J 7irnAi y - J^aM = \\J^, n A t y - J^ nAi [y + (1 - li,nhn){J ln A t y - y)) II 

<\^-ii,nhn\\\J ln Aiy-y\\- (3-14) 

Hence, in view of (|3.14j) . (|3.13p . and the inequality inf n€ N7n > £ we have 



sup {{J^AiV ~ JjnAiVll < I 1 ~ 7*,n/7»| < ~ X] ~ < +°°> ( 3 - 15 ) 



which yields (|2^|) . 

(ii) : For every y G Hi such that ||y|| < /) and every n G N, Lemma EEJRvJ and (|3T3j) yield 

2fc 



l J Tfe(' li '"A 1 )f ~ J 7«^ll < _i_ 

I11T ft n 



It II ^ 2fli n 

WiuAiV-yW < —v- 



(3.16) 



Consequently, (|2.4|) holds. 



(hi) Set = max{p + || J 7J ^0||, (/0 + H^AjOlD/T} an d let E e C Hi x Hi be as in Dehnition 13.51 

(3.17) 



It follows from [TJ Proposition 1.2] that E g n gra^4i 7^ and that 



(Vn G N) sup || J~/A,, n y ~ J-jaMI < ( 2 + 7) haus^^,™ ^) 
ll»ll<p 



Since, in view of (|3.12p . ^ ngN haus e (^4i jTl , ^4i) < +00, we conclude that (|2.4|) holds. □ 



4 Coupling schemes 

The coupling between the m inclusions in Problem 11.11 is determined by the operators (-Bj)i<i< m , 
which must satisfy (|1.6p . In this section, we describe various situations in which this property holds. 
In each case, the value of (5 in (|1.6p will be specified, as it is explicitly required in Algorithm 12.21 In 
this connection, the notion of cocoercivity (see Definition 12 .3p will play an important role. Examples 
of cocoercive operators include firmly nonexpansive operators (e.g., resolvents of maximal monotone 
operators, proximity operators, and projection operators onto nonempty closed convex sets). In 
addition, the Yosida approximation of a maximal monotone operator of index \ is X _cocoerc i ve [1] 
(further examples of cocoercive operators can be found in [67])- It is clear from (|2.5p that if T 
is x- cocoerc i ve ) then it is x _1 -Lipschitz continuous. The next lemma, which provides a converse 
implication, supplies us with another important instance of cocoercive operator (see also |37j). 
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Lemma 4.1 [16, Corollaire 10] Let (p: H — > R he a differ entiable convex function and let r G 
]0, +oo[. Suppose that V(p is r-Lipschitz continuous. Then V(p is t~ 1 -cocoercive. 

Lemma 4.2 Let L G 23 (H) &e a nonzero self-adjoint operator such that (Vx G H) (La; | x) > 0. 
T/ien L is \\L\l~ 1 -cocoereive. 

Proof. Set (Lx | x)/2. Then </? is convex and differentiable, and its gradient Vy: x i— ► Lx is 

||L||-Lipschitz continuous. Hence, the assertion follows from Lemma l4.1i □ 

4.1 Linear coupling 

We examine the case in which the operators (J3j)i<i< m are linear, which reduces (jl.6f) to 

m m 

(V(xi,...,x m ) G Hi x ••• x H m ) ^(5i(xi,...,x m ) I Xi) >/3^||L ? i (x 1 ,...,x m )|| . (4.1) 

i=i i=i 

We assume that, for every i and j in {1, . . . , m}, there exists Mjj G 23 (Hj,/Hi) such that 



■r/i 



(Vi G {l,...,m}) Hi X ••• X H m -► Hi: (xj)i<j< m t-> J2 M v x J- ( 4 - 2 ) 

3=1 



Thus, (|4.ip is equivalent to 



(V(i 1 ,... I i m )GW 1 x-x« m ) | a?«> 5Z M ' 



i=l J=l i=l 



3=1 



(4.3) 



Our objective is to determine tight values of (3 for which this inequality holds in various scenarios. 
As in the proof of Theorem 12.61 it will be convenient to let 7i be the direct Hilbert sum of the 
spaces (Hi)x<i<m with the notation <\2.7b and (|2.8j) . and to set 



m \ 

Yl Mi i x i ) 

3=1 J ; 



B-.n^H-.X^ (BiX^iKrn = ( ) . M ijXj ) . (4.4) 

Ki<m 



Proposition 4.3 Suppose that the following hold. 



(i) (3(i,j) G{l,...,m} 2 ) MijjkQ 



(ii) (V(i,j) G {l,...,m} 2 ) M ji = M 1 



l 3 



(iii) (V(xi, . . . , x m ) G Hi X • • • X Km) YT=1 £7=1 (Mij x j I Xi) > 
Then (|4.3[) is satisfied with (3 = 1/|||£?||| and, a fortiori, with 



> , 1 (4.5) 

Er=iE7=iii M ^n 2 
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Proof. It follows from (i) that and from (ii) that B* = B. In addition, (12. 7ft and (iii) imply 

that (Va: G 7i) {{Bx \ x)) > 0. Hence, we derive from Lemma 14.21 that B is |||-B||| _1 -cocoercive 
which, in view of (|4.4|) . (|2.7p . and (|2.8p . can be expressed as 



(4.6) 



mm j m m 

(V(X1,...,X TO ) €Hl X ••• X H m ) ^^(AfijXj | Xj) > JTTgTn E M ^ X < 

i=l j = l i=l j = l 

Hence, P~3j) holds with /3 = 1/|||S|||. Now take x € H such that < 1. Then, (fO|) and 

Cauchy-Schwarz yield 

mm 2 

nielli 2 = E E M ^ 

»=i i=i 

m ^ m 
m , m 

<E(Eii^n : 

4=1 V j=l 
m m 

< EE ii^ ii 2 - 
i=i i=i 

Thus, IIIBIH 2 < EI=i EjLi ll M iil! 2 and it follows from gSD that g3J holds with □ 



(4.7) 



In practice, one is interested in obtaining the tightest bound in (|4.3p . If |||B||| is known, one will 
use (3 = 1/|||S||| in Algorithm 12.21 This is for instance the case in the next proposition. In many 
situations, however, |||-B||| will be hard to compute and one can use the value supplied by (|4.5|) . 
which requires only knowledge of the norms of the individual operators (A^j)i<^j<^. 

Proposition 4.4 Let S = be a nonzero real m x m positive semidefinite symmetric matrix 
with largest eigenvalue A max . Set 



(Vi G {1, . . . , m}) Hi = H and B { : H m H: (sj-)i<3<m ^ E&- 

i=i 



(4.8) 



T/ien (g3D ZioWs wi/i /3 = 1/A„ 



Proof. Let A be the diagonal matrix the diagonal entries of which are the eigenvalues (Aj)i<j< m 
of H. There exists an m x m orthogonal matrix n = [irij] such that H = nAn*. Now set 
D: H -> H: x !-► (AjXj) 1<i<m and U : H -> H : x !-► = i Kij x j) i<i< TO - Then t/ is unitary 



and |||B||| 2 = |||C/CC/*||| 2 = |||£»||| 2 = supiii^i^E^iA 2 ! 
follows from Proposition 14.31 □ 

As shown next, equality can be achieved in (14. ip . 
Example 4.5 Set 

(Vi G {1, . . . , m}) Hi = H and B< : H r 
Then equality is achieved in (|4.ip with /3 = 1. 



A, 



Hence, the assertion 



j m 

: (Xj)i<j<m 1 * Xj / Xj 



(4.9) 
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Proof. Let (x u . . . , x m ) £ H m . Then 

m I - m 

^ \ X * ~ to ^ * 



i=l 



) = — 

to 



m 2 rn 



+E(^--E 



3=1 



i=l 



3=1 



rn . m 

^ Xi ~ to ^ 



i=l 



(4.10) 



which provides the announced identity. □ 

Our last example concerns a specific structure of the operators (iWiy)l<i,j<m- 

Proposition 4.6 For every k £ {1, . . . ,p} , let be a real Hilbert space and, for every i £ 
{1, . . . , to}, let Lki £ 23 (Hi,Gk)- Assume that mini<fc< p Yli=i ll-^fcill 2 > and set 

p 

(V(i,j)G{l,...,TO} 2 ) M ij = ^L% i L kj (4.11) 



k=l 



in (El. Then (1431) /loZds with 



P 



v^p v^m II j 



ki I 



(4.12) 



Proof. For every i and j in {1, ... , to}, (|4.1ip and Cauchy-Schwarz yield 



I Mi. 



Consequently, 



P 2 f P \2/P \ / p 

^L* ki L kj < ( ^ \\ L ki\\ \\Lkj\\ J < ( ^2\\L k i\\ 2 J ( E 



EE w 

i=l j=l 



< 



p rn 



EEii Lfci ii 2 )( EE'i Lfc ii 

k=l i=l J V k=l 3=1 



p m 



p m 



fc=l i=l 



(4.13) 



(4.14) 



On the other hand, it follows from (|4.1ip that conditions (i) (iii) in Proposition 14.31 are satisfied. 
Therefore, we derive from Proposition 14.31 that (14. 3p holds with (5 as defined in (14. 12ft . □ 

Remark 4.7 For every i £ {1, . . . , to} and n £ N, suppose that £ H (7i, Hi) in Algorithm 12.21 

say 

in 

Bi,n '■ 7~L —> Hi : (xj) 

l<jr'<m 



Then assumption (ii)(b) in Algorithm 12.21 is satisfied with z = 0. In addition, suppose that 

max > 
Ki<m ^— ' 



neN 



^HM^-MyP < +oo. 
i=i 



(4.16) 



Then assumption (ii) (a) in Algorithm 12.21 is satisfied. Indeed, let x £ 7i, i £ {1, . . . , to}, and n £ N, 
and set ftj jTl = ^/Y^jLi \\M-ij,n — Mij\\ 2 . Then, by Cauchy-Schwarz, 



(Bi >n - Bi)x\ 



E( M ' 

3=1 



rn 

<J2\\m, 



■^^i? I — ^i, ril 1 1^1 1 1 ; 



(4.17) 



where (I4.16P yields ^C ng ^ Ki, n < +°o> as desired. 
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4.2 Nonlinear coupling 



In this section we turn our attention to the determination of the parameter /3 in (jl.6p when the 
operators (-Bj)x<j<m ar e nonlinear. Our first model is a nonlinear version of Proposition 14,61 



Proposition 4.8 For every k G {1, . . . ,p}, let Q k be a real Hilbert space, let /3 k £ ]0,+oo[, let 
Tf.: Gk — > Qk be f3k-cocoercive, and, for every i £ {1, ...,m}, let L k i S 23 (TLi,Gk)- Assume that 
mini< A .< p ^^ 1 \\L ki \\ 2 > and set 



P / m \ 

(Vt e {1, . . . , m}) Bi-.HiX---x H m Hi : (^)i<i< m ^ L* ki T k f ^ L^x,- j . 

fc=i m=i ' 



T/ien (11.61) ZioWs wrai/i 



(3 = — vain 



ki I 



(4.18) 



(4.19) 



Proof. For every i S {1, . . . , m}, let Xi and be points in Hi. It follows from (|4.18p . (|4.19p . and 
the convexity of || • || 2 that 



^2 {Bi(xi, . . .,x m ) - Bi(yi, ...,y m )\xi- yi) 

m p j / / m 
= YY\ Lk i[ Tk [Y Lk i X i 



i=l 



i=l k=l 
m p 



Tk[^L kj y^j 



x% yi 



L k i{xi - Vi) 



IIL b» I / III \ / III \ 

YY\ Tk (Y Lk 3 x i )- Tk (Y Lk i y j ) 

i=l k=l * ^ j=l ' ^j=l ' 

P i / m \ / m \ m in 

Y{ Tk [Y Lk 3 x i )~ Tk (Y Lk 3 y i ) Y LkiX * ~Y L 

k=l ^ M=l ' M=l ' i=l i=l 



> Y fa Tk (Y Lk i x i ) - Tk ( Y Lk iyj 



i=i 



fc=i 
p 

m p 



i=i 



ll^fcill 2 T kiY Lk i x i ) ~ Tk ( Y Lk i y i 

8=1 ^ 7=1 ' ^ 7 =1 



m v / / in \ / in \ \ 

^ f 3 Y p Y L *ki( Tk [Y Lk i x i ) - Tk ( Y Lk iyj ) ) 

i=l k=l ^ i=i i=i 

mp i m \ p , m 

>pY Y Lk i Tk (Y Lk 3 x 3j-Y Lk MY Lk 3y 

i=l k=l ^j=l ' k=l ^3=1 



(4.20) 



which establishes the inequality. □ 



Remark 4.9 Suppose that T k = Id in Proposition 14.81 Then the operators (-Bj)i<j< m of ()4.18|) 
are simply those resulting from Proposition 14.61 However, since (5 k = l, the bound given in (I4.12|) 
is sharper than that given in (|4.19|) . 
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Corollary 4.10 For every k £ {1, ...,p}, let Gk be a real Hilbert space, let £ ]0, +oo[, let 
(fk- Gk — > ^ fre a T^-Lipschitz-dijferentiable convex function, and, for every i £ {1, . . . ,m}, let 
Lki £ 25 {TLi,Qkj- Assume that mini<fc< p Ya= i ll-^feill 2 > anc ^ se ^ 



(Vi 6 {1, . . . , m}) Si : Hi x • • • x H TO Hi : (xj)i<j<r 
TTien (-Bj)i<j< m satisfies (|1.6p wit/i 



fc=l ^ 7=1 ' 



/? = s; • (4.22) 



p max r fc \\L ki \\ 
Kk<p ^ 
F i=l 



Proof. Lemma I4TT1 asserts that, for every k £ {1, ... ,p}, = Vfk is r fc -cocoercive. The result 
therefore follows from Proposition 14.81 □ 

Example 4.11 (saddle point problems) For every k £ {1, . . . ,p} and I £ {1, . . . , q}, let Gk and 
JCi be real Hilbert spaces, let £ ]0, +oo[, let m £ ]0,+oo[, let ^: Sj. -> 1 be a Tfc-Lipschitz- 
differentiable convex function, let ipi : ICi — > R be a K^-Lipschitz-differentiable convex function. 
Furthermore, for every i £ {1, . . . , m}, let 3^ and Zi be real Hilbert spaces, let Fi : 3j © Z% — > 
[— oo, +oo] satisfy [(I)| and (ii) in Example l3.3l let Lfcj G 25 (2j, £/&) and Mu £ 25 (3^, /Q). It is assumed 
that min^fc^p YliLi ||^fci|| 2 > and that mini<;< g Ya=i ll-^z«ll 2 > 0- Consider the problem 



maximize minimize 

yi&yi,--;ym&ym Zl£Zl,...,Z m £Zri 



m q / m \ P / m \ 

Hvh **) - X) ^ ( J2 M ^ ) +12^412 LkiZi ) • ( 4 - 23 ) 

i=l 2=1 ^ i=l ' k=l ^ i=l ' 



(4.24) 



Now set _ 

f £ a : 3^i x • • • x y m -»■ # : (yj)i<,-<m ^ ELi ( E7=i 

[B i2 : Z\ X • • • X Z m ->■ Zi \ {Zj)l<j< m !->■ Yfk=l L ki^ Vk{Y!j=l L kjZj) , 

and 

A = ii || 2 and 02 = fi r M2 - ( 425 ) 

g max ^ =1 ||Mh||^ p max r fc ^ • x ||L fej -|^ 

We derive from Corollary 14.101 that, for every (yi, . . . , y m ) and (y 1 , . . . ,y m ) in 3^i X • • • x y m , 

m 

^2(Bn(yi,...,y m ) - B il (y 1 , . . . ,y m ) | ^-y.^ > 

i=l 

m 

Pi ^2 ll^i (yi' - • -'Vm) - Bii(y 1} .. ■ ,y m )\\ y ., (4.26) 
i=i 

and that an analogous inequality is satisfied by Bi 2 with fa. On the other hand, using minimax 
theory [57], we can cast (I4.23j) in the form of (|3.5p where, for every i £ {1, . . . , m}, Tti = yi © Zi 
and 

Bi = (Bii,B i2 ): (yj,Zj)i<j<m (Bil(yi, ■ ■ ■ , Dm), B i2 (z 1 , . . .,z m )). (4.27) 

Altogether, it follows from Example 13.31 that (|4.23p is a special case of Problem 11.11 in which 
{Bi)i<i< m satisfies (11.6P with f3 = min{/?i, f} 2 }. 
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5 Coupling evolution inclusions 



Evolution inclusions arise in various fields of applied mathematics |41|, 160] . In this section, we 
address the problem of solving systems of coupled evolution inclusions with periodicity conditions. 
The notation and definitions introduced in Example 13.41 will be used. 



5.1 Problem formulation and algorithm 

Problem 5.1 Let (Hi) \<i<m be real Hilbert spaces and let T E ]0, +oo[. For every i E {1, . . . , m}, 

set 

Wi = {xEe([0,r];Hi)nW 1 ' 2 ([0 J T];H i ) I x(T) = x(0)}, (5.1) 

let fj E To(Hj), and let B« : Hi x • • • x H m — ► H«. It is assumed that there exists (3 E ]0, +oo[ such 
that 

(V(xi, . . . ,x m ) E Hi x • • • x H m )(V(yi, ... ,y m ) G Hi X ••• X H m ) 

m m 

(Bi(xi, ... ,x m )- Bj(yi, ... ,y m ) | Xf — yi) Hj > (3^ || B i( x i> • • • ,x m ) - Bj(yi, . . . ,y m )|| H .. (5.2) 

1=1 1=1 

The problem is to 

find x\ E Wi, . . . , x m E W m such that 

(Vi E {l,...,m}) 0E^(t) + 9f i (x i (t)) + B i (xi(t),...,a; m (t)) a.e. on ]0,T[, (5.3) 

under the assumption that such solutions exist. 

Algorithm 5.2 Fix e E ]0, min{l, /?}[, (7 n ) nS N in [s,2f3 — e], and (A n ) ng pj in [0,1 — e]. Let, for 
every n E N and every i E {1, . . . , m}, yi^ n be the unique solution in W% to the inclusion 

(*)) + &i, n (*)) 

7n 

G + 9fi(y i!n (t)) + e iin (t) a.e. on ]0, T[ (5.4) 

and set 

^i, n+1 = \,n%i,n + (1 — \,n)yi,n (5-5) 

where, for every i E {1, . . . , m}, the following hold. 

(i) Xi, € W^ao^Hi). 

(ii) (6i, n )neN an( i (ej,n)nGN are sequences in L 2 ([0,T]; Hi) such that 



J2\ f \\Kn(t)\\ 2 H dt < +00 and ^ W I ||e iin (f)|| 2 ,dt < +00. (5.6) 

(iii) (A l>n ) ne N is a sequence in [0, 1[ such that YlneN \\i, n - A n | < +00. 

In (|5.4p . bi tTl (t) models the error tolerated in the computation of Bj(xi iTl (t), . . . ,3?m,n(^))> while 
ei t n(t) models the error tolerated in solving the inclusion with respect to <9fj(yj >n (i)). 
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5.2 Convergence 



Theorem 5.3 Let ((xj in ) nS N)i<i<m be sequences generated by Alaorithm \5.£[ Then, for every i G 
{1, . . . , to}, (xj.n)neN converges weakly in W ,2 ([0, T]; Hj) to a point Xi G Wj, and (x{)i<i< m is a 
solution to Problem \5. 1[ 



Proof. For every i £ {1,... , m}, set Hi = L 2 ([0, T]\ Hj) and 
A C . Hi -» 2 W » 

fj-u G I u(t) G x'(i) +9fi(a;(t)) a.e. in ]0,T[|, if x G Wf, (5.7) 
[0, otherwise. 

Let us first show that the operators (^4j)i<i< m are maximal monotone. For this purpose, let i G 
{1, . . . ,m}, (x,u) G graAj, and (y,v) G graAj. It follows from (|5.T[) that, almost everywhere on 
]0,T[, u(t) — x'(t) £ dfi(x(t)) and v(t) — y'(t) G dU(y(t)). Therefore, by monotonicity of dU, we have 

((u(i) - x'(t)) - (v(t) - y'(i)) I x(t) - y(t)) Hi tft > 0. (5.8) 



Hence, 



(u - v I x - y) = I (u{t) - v(t) I x(t) - y{t)) H dt 







T ({u(t) - x'(t)) - (v(t) - y\t)) I x(t) - y{t)) H dt 



dt 



+ J (x'(t) - y'(t) I x(t) - y(t)) Hi 
1 f T d\\x(t)-y(t)\\^ 



> - / —— — '-^±dt 
~ 2 J dt 

= i(||x(T)- y (T)|| 2 Hi -||x(0)- y (0)|| 2 H J 

= 0. (5.9) 

Thus, Ai is monotone. To prove maximality, set gj = (1/2)|| • ||^. + fj. Then g, G ro(Hj) and 
dgi = Id +9fj. Moreover, since fj G Fo(Hj), it is minorized by a continuous affine functional, say 
fi > (• I v) H . + rj for some v G Hj and 77 G K. Now, let y G domfj = domgj and take (x, u) G gradgj. 
Then (jl.lOp and Cauchy-Schwarz imply the coercivity property 

( X ~V I u )h, > gi(x) -gj(y) 

ll x lk " ||x|| Hj 

_ ll x lk f*(x) -gi(y) 



2 IMlm 

" 2 11 l|H ' ||x|| Hi 

— > +00 as I |x| I - * +00. (5.10) 
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Therefore, [251 Corollaire 3.4] asserts that for every w G Hi there exists z G Wj such that 

w(t) G z'(t) + d gi (z(t)) = z'(t) + z{t) + dfi(z(t)) a.e. on ]0, T[ , (5.11) 

i.e., by (j5.7|) . such that w — z G This shows that the range of Id +Ai is Hj and hence, by 

Minty's theorem |11| Theorem 3.5.8], that Ai is maximal monotone. 

Next, for every i G {1, . . . , m} and every (xi, . . . , x m ) G Hi x • • • x H OT , define almost everywhere 

B i (x 1 ,...,x m ): [0,T] -» Hi 

t i ^ Bj(xi(t), . . . ,rc m (i)). 



(5.12) 



Now let (aci, . . . , x m ) G Hi x • • • x Tt m and set (Vi G {1, . . . , m}) bj = Bj(0, . . . , 0). Then it follows 
from (|5.2p and Cauchy-Schwarz that, almost everywhere on [0, T], 

m m 

pJ2 11^(^1 (*)> • • • ,zm(t)) - bjllS. < ^ (B,(xi(t), . . . ,x m (i)) - bj I x,(i) - 0) H? 



j'=i 



^ ^2\\Bji x x(t), . . . ,x m (t)) ||a;j(t)llHj 

3=1 



< 



J2\\Bj{xi(t), ■ ■ ■ ,x m {t)) - bM 



i=i 



(5.13) 



Therefore, for every i G {1, . . . , m}, 

||Bi(xi, . . . ,x m )(t)|| 2 Hi < 2(||b<||S. + \\Bi(xi, . . .,x m )(t) - bif H .) 

< 2(\\b i f H . + l|B,(xi(t), . . .,x m (t)) - bj\\\ 



3=1 



( 1 - \ 

^^II^II^ + ^Eh^WIIh.J a - e - on M. ( 5 - 14 ) 



which yields 



£ WBiixx, . . .,x m ){t)\\l4t < 2T||b i || 2 Hi + ^jt 



h 3 II > 



(5.15) 



so that we can now claim that B{ : Hi x • • • x H m -> L 2 ([0,T];Hi) = Hj. In addition, upon 
integrating, we derive from (|5.2p and (|5.12[) that, for every (yi, . . . , y m ) G Hi x • • • x H m , 

m m 

• • ■,%) - -Bi(yi, •• • ,2/m) I - > \\Bi(xi, . . .,x m ) - B^yx, . . .,y m )\\ ■ (5.16) 



i=i 



We have thus established (11.61). 



i=l 



Let us now make the connection between Algorithm 15.21 and Algorithm 12.21 For every n G N and 
every i G {1, . . . ,m}, it follows from (|5.4p . (15. 7h . (j5.12|) . and the maximal monotonicity of that 
?/i in is uniquely defined and can be expressed as 



J~1n,Ai ( 2^,71 (-^i (^l,n i • • • j 2?m,n) ~l~ fti,n) J "I - Q 



(5.17) 
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where 



^i,n — J'Yn.Ai r )n&i,n~\~%i,n ^ni y Bi{x\ n) ...,X mn ^^bi r ^j Jy n Ai(^Ei,n 'Jn{^^iixi,n: • • • j •Em,n)~l _ ^i 1 n) j j 

(5.18) 

and we therefore derive from (15.41) and (15.51) that 



■^i,n+l — -^n^i.n "I - (1 ^i,n) ^<-^7 n A; (^i,n In (-^t ! • • • j ^m,n) ^i,n) J "I - ^i,n^ • (5.19) 

We observe that (15.191) derives from (12, 3h . where ^4j jn = A, and -Bj jTC = -Bj. On the other hand, for 
every i G {1, . . . , m}, by nonexpansivity of the operators (J<y n A^)n&ii we deduce from (|5.18p and 
flOD that 

neN neN neN 

As a result, all the hypotheses of Algorithm 12.21 are satisfied and hence Theorem 12.61 asserts that, for 
every i S {1, ... , m}, (xj, n )neN converges weakly in Hi = L 2 ([0,T]; Hj) to a point Xj, and (xj)i<i< m 
satisfies 

(Vi G {1,... ,m}) G AjXj + i?j(xi, . . . , x m ). (5.21) 

Accordingly, 

0"= max sup n || < +oo (5.22) 

l<i<m neN 

and (Vi £ {1, . . . , m}) Xj G domAj C Wj. Moreover since, in view of (|5.7p and (I5.12|) . (I5.2ip reduces 
to ()5.3p . (xj)i<j< m is a solution to Problem 15.11 

To complete the proof, let i G {l,...,m}. To show that (xi,n)rceN converges weakly to Xj in 
W ,2 ([0, T]; Hj), it remains to show that (a>j n )neN converges weakly to x\ in L 2 ([0,T]; Hj). We first 
observe that (xj in ) ne N 

lies in W^QO.T]; Hj). Indeed, it follows from <$5/fl) that 
(Vn G N)(V* G Wj) J 7 „ A 2 G dom( 7n ^) C W< C W^ 2 ([0,T}; Hj). (5.23) 
As a result, we deduce from (15.18P that (aj, n )neN lies in W 1,2 ([0, T]; Hj). On the other hand, 



by construction, (yi, n ) n eN lies in Wj C W ' 2 ([0, T]; Hj). In view of (|5.5p and (i) in Algorithm 15.21 
(xj in ) nG pj is therefore in W^ 1,2 ([0, T]; Hj). Next, let us show that (x[ n ) n eN is bounded in L 2 ([0, T]; Hj). 
To this end, let n G N and set 

m,n(t) = Xi,n{ ^ ~ Vi,n ® - Bi(xi, n (t), x m>n {t)) - bi, n (t) - y\ n (t) - e i>n {t) a.e. on]0,T[. (5.24) 

In 



Then we derive from (|5.4p that 

Wi,n(t) G dfi( yi , n (t)) a.e. on ]0,T[. (5.25) 
Hence, since Wi^ n G Hi, it follows from [251 Lemme 3.3] that 

d{U ° y ^ ){t) = (w iin (t) | yUt)) Hi a.e. on ]0, T[ . (5.26) 
On the other hand, since yi >n G Wj, we have yj jn (T) = j/j in (0). Therefore 



^ <w*,n(t) I ViMn^ = [ d{il0y ^ n){t) dt = fi(m,n(T)) - fi( W ,„(0)) = 



(5.27) 
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and, furthermore, 
fT 



o 



(t) I l/i, n (t)) H .dt 



-at — 



o 



dt 



0. 



We deduce from QSgTD , <^M), and ([OH]) that 



\ In 



T 



(*)) I y'i, n {t)) dt 



ii 



T 



\Vi,nW\\Hi 



dt 



<ei,n(t) I y' itn (t)) H dt. 



(Knit) I yi, n (t)) H dt- 

Thus, using Cauchy-Schwarz, the inequality 7„ > e, and (|5.12p . we obtain 

|| II — II ^i,n || (^l,n, • • • , £m,n) || "I - || ^i,n || "I - || &i,n || ^ II J/i,nll * 

In turn, it follows from (|5.5p that 

II "^n+l II — ^i,n || 1 1 "I - (1 ^i,n) \\^i,n I "I - ||-^i (^"l,ri) • • • > ■^m,n) || "I - || &i,n || "I - || ^i,r. 

On the other hand, arguing as in (|5,15p . we derive from (|5,22|) that 



2ma 2 



|Bi(x 1 ,„,... > x m ,n)|| < W2r||bi||2. + — - < v^HbillHi + V2m 



■ (7 
0" 



Hence, using (ii) in Algorithm 15.21 we derive by induction from (I5.3ip that 



(5.28) 



(5.29) 



(5.30) 



(5.31) 



(5.32) 



||:r; n || < max <^ \\x' { ||, - + \/2T||bi|| Hi + VZm ^ + sup (||6j,fc|| + \\ei,k\\) >■ (5.33) 
{ ' e p k& J 

This shows the boundedness of (x^ n ) n eN in L 2 ([0, T]; Hj). Now let z be the weak limit in 
L 2 ([0, T]; Hi) of an arbitrary weakly convergent subsequence of (x' in ) n ^n. Since (xi )n )neN converges 
weakly in L 2 ([0, T]; Hj) to Xj, it therefore follows from |66[ Proposition 23.19] that z = x\. In turn, 
this shows that (x' in ) n£ ^ converges weakly in L 2 ([0, T]; Hj) to x\. □ 



6 The variational case 

In this section, we study a special case of Problem 11.11 which yields a variational formulation that 
extends (jl.5p . This framework can be regarded as a particular instance of Example 14.111 



6.1 Problem formulation and algorithm 

Recall that, for every / £ Yq((H) and every x S TL, the function y i— ► f(y) + ||x — y|| 2 /2 admits 
a unique minimizer, which is denoted by prox^-x. The proximity operator thus defined can be 
expressed as proxj = Jqj [51]. 
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Problem 6.1 Let (Hi) i<i<m and (Qk)i<k<p be real Hilbert spaces. For every i £ {1, . . . ,m}, let 
fi £ To(Hi) and, for every k £ {l,...,p}, let r k G ]0,+oo[, let (fk- Gk -> R be a Tfc-Lipschitz- 
differentiable convex function, and let L k i £ 23 (Hi,Q k ). It is assumed that mini<fc< p ||Lfci|| 2 > 
0. The problem is to 

m p , m s 

minimize fi(xi) + (f k ( L ki Xi ) , (6.1) 

xi£Hi,...,x m &i. m ^— * f— ' V / 
«=1 fc=l x i=l ' 

under the assumption that solutions exist. 
Algorithm 6.2 Set 

P= l — • (6-2) 



m 

p max. r k ^2 \\ L k n ' 2 



i< /,-</) " ' ■ ' 



Fix e £ }Q,mm{\,(3}[, (7„)n6N in [e, 2/3 - e], (A n ) neN in [0, 1 - e], and (zi )0 )i<i< m £H\X ■ ■ ■ xH r 
Set, for every n G N, 

%l,n+l = Ai jra Xi jn + 

fc=i i 



(1 — A m 

"m,n J J "T &m,n I ; 



(6.3) 

where, for every i G {1, . . . , m}, the following hold. 

(i) (fi,n)neN are functions in Tq(Hi) such that 

(VpG]0,+oo[) ^2 sup ||prox 7n/in y-prox 7n/i y|| < +oo. (6.4) 

nGN Hfll^ 

(ii) (ai, n )ngN and (&i, n )neN are sequences in Hi such that X^neN \\ai,n\\ < +oo and X neN ||&i, n || < 
+oo. 

(hi) (Xi : n)n€N is a sequence in [0, 1[ such that X^neN \^i,n ~ < +°o. 

Theorem 6.3 Let ((xi jn ) n ^)i<i< m be sequences generated by Algorithm \ 6.SX Then, for every 
i £ {1, . . . ,m}, (xj, n )neN converges weakly to a point G Hi, and (xi)i<i< m is a solution to 
Problem\6.1[ 



Proof. Problem 16.11 is a special case of Problem ll.il where , for every i £ {1, . . . , m}, 

P / m \ 

Ai = dfi and Bi: (xj)\<j< in 

fc=l ^ 7 = 1 ' 
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Indeed, define 7i as in the proof of Theorem 12.61 and set 

m 

f:H-> ]-oo,+oo] : (xi)i<i< m i-> y~]fi(xj) (6.6) 

i=l 

and 

l<i<m 

k=l ^ 8=1 ' 

Then / and g are in To('H) and it follows from Fermat's rule and elementary subdifferential calculus 
that, for every (x±, . . . , x m ) G "H, 

(xi, ... ,x m ) solves dHUJ) 44> (0, ...,0) G 5(/ + £jf)(xi,...,x m ) 

(0, . . . , 0) G «9/(xi, ...,x m )+ Vg(xi, x m ) 



v / m \ 

k=i ^i=i ' 



^ (Vi G {1,... ,m}) G 
44> (Vi G {!,..., m}) G AiXi + Bi(xi,...,x m ). (6.8) 



In addition, Lemma 14.11 asserts that, for every k G {l,...,p}, is r^-cocoercive. In turn, 

we derive from Corollary 14.101 that the family (Bj)i<i< m in (|6.5p satisfies (|1.6p with /3 as in (|6.2p . 
Setting 

(ViG{l,...,m})(VnGN) A,n = dfi !n and B i>n = B h (6.9) 

we deduce from (I6.4h that Algorithm 16.21 is a particular case of Algorithm 12.21 Altogether, Theo- 
rem [63] follows from Theorem 12.61 □ 



6.2 Applications 

Let us consider some applications of Theorem 16.31 starting with a game-theoretic interpretation of 
Problem 16.11 



Example 6.4 (coordinated games) Consider a game with m players indexed by i G {1, . . . , m}. 

The strategy Xi of the ith player lies in the real Hilbert space Hi and his individual utility is 
modeled by a proper upper semicontinuous concave function h^. 7ii — > [— oo,+oo[. In the absence 
of coordination, the goal of each player is to maximize his own payoff, which can be described by 
the variational problem 

m 

maximize > hi(xj). (6.10) 
en 

1=1 

A coordinator having a global vision of the common interest of the group of players (say, a benevolent 
dictator |52j) imposes that, instead of solving the individualistic problem (I6.10|) . the players solve 
the joint equilibration problem 

m 

maximize > hi(xi) + g(x\, . . . ,x m ), (6-11) 

2=1 
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where g : @ i=1 — > R is a Lipschitz-differentiable concave utility function that models the collec- 
tive welfare of the group. A finer model consists in considering p subgroups of players and writing 
g = Yl P k=i 9k-> where the payoff g k of subgroup k 6 {1, . . . ,p} can be expressed as 

9k - O&l, • • • > x m ) h-> Vfc f LkiXi ) ' ( 6 ' 12 ) 

where Vfc is a Lipschitz-differentiable concave function on a real Hilbert space Qk and where, for 
every i G {1, . . . , m}, Lki G B (Wj, £/&). In this model, player i is involved in the activity of subgroup 
k if Lki 7^ 0. Upon setting /, = —h% for every i E {1, . . . , m} and = — ^fc for every k £ {1, . . . 
we recover precisely Problem 16. 11 Let us notice that a solution (xi, . . . , x m ) to (|6.1ip - (|6.12p can be 
interpreted as a Nash equilibrium of the potential game [50] in which the payoff of player i in terms 
of the strategies of the remaining m — 1 players is given by 



X% I ► hi(x>l 



P / m \ 

+E^(E L ^)' (fU3) 

fc=l V 7 = 1 ' 



In this framework, Theorem 16.31 provides a numerical construction of a Nash equilibrium of the 
game, and Algorithm 16.21 provides a dynamical model for the interaction between the players. At 
iteration n of Algorithm 16.21 each player % aims at maximizing the utility given in (|6,13p . This is 
carried out by the proximal step (j6.3|) . which is a relaxed version of the exact proximal step 



Yl L kt^^k f Y L kj X J,n ) ) • 

k = l ^ 7=1 ' ' 



Xi,n+i = prox 7n/ . I x i>n - 7n 2^ L ki^fk I L kj Xj. n I , (6.14) 



in which the function f, L is replaced by an approximation fi tU , and some errors ai iTl and bi <n are 
tolerated in the numerical implementation of prox 7n j. and (V</3fc)i<fc< p , respectively. The last 
ingredient of this step concerns risk aversion and is modeled by the relaxation parameter A^ n . 
When Aj„ = 0, player i makes a full proximal step; otherwise, his step is more heavily anchored 
to his current position Xi n due, for instance, to uncertainty concerning the next performance of his 
payoff. Let us note that, in the absence of coordination {ipk = 0) the dynamics of each player would 
just evolve independently through pure proximal iterations. The coordinator modifies the current 
strategy x^ n by adding to it a component in the direction of the gradient of the collective utility, 
namely — 7 n YX=i ^ki^ Vfc(!Cj=i ^kj x i,n)- in this simultaneous dynamical game, the players choose 
strategies in a decentralized fashion and without knowledge of the strategies that are being chosen 
by other players. 

Example 6.5 (2-agent problem) In Problem 16. 1\ set m = 2 and p = 1. Then (16. ip becomes 

minimize fi(x\) + / 2 (x 2 ) + ^>\(L\\X\ + L\ 2 x 2 ). (6.15) 

Now suppose that (f \ is the Moreau envelope of a function ip £ Tq(Qi), i.e., 

<pi:x^ inf i/j(y) + -\\x - y\\g . (6.16) 
y&Qi £ 



Then V(fi = Id — prox^, has Lipschitz constant t\ = 1 [51]. Let us employ the simple form of (|6.3p 
in which A n = 0, Ai >n = 0, A 2 , n = 0, a 1>n = 0, a 2<n = 0, = /i, / 2 ,„ = /2, h,n = 0, and 6 2 ,n = 0, 
namely 

jxi,„+i =prox 7n/i (xi in +7„L* 1 (prox^-Id)(LnXi >n + Li2X 2i „)) ^ 
1 x 2 , n +i = prox 7n/2 (x 2 , n +7„L^ 2 (prox^ -Id)(LnXi >n + L 12 x 2 ^ n )). 
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Theorem [fL3] asserts that, if (7 n )neN lies in [e^dlLnf + HLiall 2 )- 1 -^ for some arbitrarily small 
e G ]0, +oo[, then ((xi jn , x 2j „)) ne N converges weakly to a solution (xi,x 2 ) to (|6,15|) , In particular, 
if ip is the indicator function of a nonempty closed convex subset C of Q\, then (|6.15p and (|6.17|) 
become respectively 

minimize fi(x{) + / 2 (x 2 ) + l;d^(L n xi + L 12 x 2 ) (6.18) 

X 1 eH.l,X2e:H2 2 



and 

\xi, n+1 = prox 7n/l (xi >n + 7 n LJ 1 (P c - - Id)(LnXi >n + Li 2 x 2 ,n)) 

| > X2 J n+l = P roX 7 n / 2 { x 2,n + j n L* 12 (P c - Id )(L U Xl tn + L 12 X 2 , n )) . 

A further special case of interest is when C = {0}, meaning that (|6.18p reduces to (|1.5p . i.e., 



(6.19) 



minimize /i(xi) + f 2 (x 2 ) + -||£nxi + L^^Hg, , (6.20) 



and that (|6.17p assumes the form 



xi, n+ i = prox 7n/l (xi, n - 7„LJ 1 (LnXi in + L 12 x 2 , n )) 

X 2)U+ i = prOX 7n/2 (x 2 ,n ~ ln^\ 2 (^llXl,„ + L 12 X 2 ,n))- 



(6.21) 



In [5], (|6.20p was approached via an inertial alternating proximal algorithm. Finally, if we further 
specialize (|6.20p by choosing 7ii = TL 2 = Q\ and L\\ = Id = — L\ 2 , then (|6.20p reduces to (|1.2j) . 
which was first considered in [lj. In this case, upon setting j n = 1/2 in (|6.2ip we obtain the parallel 
proximal algorithm 

I xi, n+ i = prox /l/2 ((xi, n + x 2 , n )/2) ^ 
\x 2 , n+ i = prox /a/2 ((xi jn + x 2j „)/2) . 

In view of the above analysis, the sequence ((xi jn , x 2jn )) ng N thus generated converges weakly to a 
solution to (|1.2p . In [Tj, the same conclusion was reached for the sequential algorithm (see also [9] 
for an alternative algorithm with costs-to-move) 

fxi, n+ i =prox /l x 2 , n 
\x 2i „+i = prox /2 xi >n+ i. 

Example 6.6 (traffic theory) Consider a network with M links indexed by j G {1, . . . , M} and 

./V paths indexed by / G {1, . . . , N}, linking a subset of Q origin-destination node pairs indexed by 
k G {1, . . . , Q}. There are m types of users indexed by i G {1, . . . , m} transiting on the network. 
For every i G {1, . . . , m} and / G {1, . . . , N}, let £ M be the flux of user i on path / and let 
x i = (£,u)i<i<N be the flow associated with user i. A standard problem in traffic theory is to find 
a Wardrop equilibrium [64] of the network, i.e., flows (xj)i<j< m such that the costs in all paths 
actually used are equal and less than those a single user would face on any unused path. Such an 
equilibrium can be obtained by solving the variational problem \21\ \54 



M nhj(xi,...,X m ) 

minimize I 4>j(h)dh, (6.24) 

xieCi,..., x m £C m ~^Jo 

where 4>j : M — > [0,+oo[ is a strictly increasing r-Lipschitz continuous function modeling the cost 
of transiting on link j and hj{x\, . . . ,x m ) is the total flow through link j, which can be expressed 
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as hj(xi, . . . , x m ) = Y^iLiiLxi) 1 ej, where ej is the jth canonical basis vector of M M and L is 
an M x N binary matrix with jZth entry equal to 1 or 0, according as link j belongs to path 
I or not. Furthermore, each closed and convex constraint set Cj in (|6.24[) is defined as Cj = 
{(m)l<l<N G [0,+oo[ w | (Vfc G {1, . . .,Q}) J2ieN k Vl = $ik}, where ^ N k C {1, . . . , N} is the set 
of paths linking the pair k and <5jfc G [0, +oo[ is the flow of user i that must transit from the origin 
to the destination of pair k (for more details on network flows, see [581 159j ). Upon setting 



M 

(pi : R M 



problem (|6.24p can be written as 



minimize 

xl m N ,...,X m & 



■ (vj)i<j<M 4>j{h)dh, (6.25) 

i=i Jo 

m / m v 

^i^{x i ) + i Pl (j2Lx i \. (6.26) 

i=l ^ i=l ' 



Since (pi is strictly convex and r-Lipschitz-differentiable, (|6.26p is a particular instance of Prob- 
lem O with p = l,Q 1 = R M and (Vi G {1, . . . ,m}) ^ = l w , /j = and L M = L. Accordingly, 
Theorem 16.31 asserts that (16.26H can be solved by Algorithm 16.21 which, with the choice of parameters 
7„ = 7 G ]0, 2/t[, A» )n = 0, A n = 0, a ijn = 0, and 6 i)7l = 0, yields 



(Vi G {1, . . . , m}) Xi, n+1 = P<- %i,n ~ jL 



'Y^Lx ijn V... ,<j> m [ ^Lx iin ) ) J. (6.27) 
i=i ' m=i ' ' ' 



In the special case when m = 1 the algorithm described in (|6.27p is proposed in [22J. Let us note 
that, as an alternative to (pi in f)6.25|) . we can consider the function 

M 

<pi : R M -> M: (^-)i<i<M ^ ^iC^)' ( 6 - 28 ) 

i=l 

under suitable assumptions on (4>j)i<j<M- In this case, (|6.26p reduces to the problem of finding the 
social optimum in the network [59], that is 

At 

minimize ^ hj(xi, . . . , x m )4>j (hj(xi, . . . , x m )) , (6.29) 



which can also be solved with Algorithm 

Example 6.7 (source separation) Consider the problem of recovering m signals (xi)i<i< m lying 
in respective Hilbert spaces {Ji.i)i<i< m from p observations (z k )i< k < p lying in respective Hilbert 
spaces {G k )i< k < p - The data formation model is 

m 

(Vfc G {1, . . . ,p}) z k = ^2L ki Xi + w k , (6.30) 

i=l 

where L k i G H (7ii,G k ) and where w k G Q k models observation noise (see in particular [23, 43J). In 
other words, the objective is to recover the original signals (xi)i<i< m from the p mixtures {z k )i< k < p . 
This situation arises in particular in audio signal processing, when p microphones record the su- 
perpositions (<Zfc)i<fc<p of m sources (xi)i<j< m that have undergone linear distortions and noise 
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corruption. Let us note that the same type of model arises in multicomponent signal deconvolution 
problems [3J |H] . A variational formulation of the problem is 



minimize , 

en m 

I 



P / m \ 

^fi(xi) + ^2D k (^2L ki Xi,z k j. (6.31) 
i k=i ^ i=l ' 



In this formulation, each function /j G To(7ii) models some prior knowledge about the signal x^. 
On the other hand, each function D k : Q k x Q k — > [0, +oo[ promotes data fitting: it vanishes only on 
the diagonal {(z,z) | z G and, for every z G D k (-,z) is convex and Lipschitz-differentiable 
(for instance, D k can be a Bregman distance under suitable assumptions I27|. and in particular 
the standard quadratic fitting term D k : (y,z) t— > \\y — z\\g). It is clear that (|6.3ip is a special 
realization of Problem 16.11 (with ip k = D k {-, z k ) for every k G {1, . . . ,p}) and that it can therefore 
be solved via Algorithm 16.21 



Example 6.8 (image decomposition) A standard problem in image processing is to find the 
decomposition (xi)i<i< m of an image x = X^Li x % m some Hilbert space H., from some observation 
z. When m = 2, a common instance of this problem is the geometry /texture decomposition problem 
[12} I14j. The variational formulations studied in these papers are special instances of the problem 



m m 

minimize > fdxj) H — z — > 



=1 



(6.32) 



where (fi)\<i< m are functions in ro(W). The first term in the objective is a separable function, the 
purpose of which is to promote certain known features of each component Xi, and the second is a 
least-squares data fitting term. As shown in [34], for m = 2, (|6.32p can be solved by alternating 
proximal methods, which produce weakly convergent sequences. In [13) . a finer 3-component model 
of the form (I6.32h was investigated in TL = M , and a coordinate descent algorithm was proposed 
to solve it. This algorithm, however, has modest convergence properties, and it was proved only 
that the cluster points of the sequence it generates are solutions of the particular finite dimensional 
problem considered there. By contrast, since (16. 32ft is a special case of Problem 16.11 (with Hi = Ti., 
k = 1, <f i = \\z — -|| 2 /4, and Ln = Id), we can derive from Theorem 16.31 an iterative method the 
orbits of which are guaranteed to converge weakly to a solution to (|6.32p , under the sole assumption 
that solutions exist. For instance, for m = 3, (|6,2p yields [3 = 2/3. Taking for simplicity 7„ = 1, 
A n = 0, and, for i G {1, 2, 3}, A i>ri = 0, a i;n = 0, f i>n = fa, and b i)Tl = 0, (fOl becomes 

Xl,n+1 = P rox /i {(Z + Xl,n ~ %2,n ~ ^3,n)/2) 

X2,n+i = prox /2 ((z - xi tn + X 2 , n ~ X3 >n )/2) (6.33) 
X^n+l = PrOX /3 [(z - X^ n - X 2>n + X^ n )/2) . 

Let us note that, since Theorem 16.31 allows for more general coupling terms than that used in (|6.32[) . 
more sophisticated image decomposition problems can be solved in our framework. 

Example 6.9 (best approximation) The convex feasibility problem is to find a point in the 
intersection of closed convex subsets (Cj)i<j< m of a real Hilbert space Ti p21[29]. I n many instances, 
this intersection may turn out to be empty and a relaxation of this problem in the presence of a 
hard constraint represented by C\ is to |32j 



minimize 

xieCi 



^ m 

-J2"id 2 Ci (xi)> ( 6 - 34 ) 
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where (uji)2<i< m are strictly positive weights such that max2<i< m u;j = 1. Since, for every i G 
{2, . . . , to} and every x\ G C\, d 2 c .(x\) = min^gQ ||xi — x,i\\ 2 , (|6.34j) can be reformulated as 



m— 1 



minimize ^ ]P ^fc+ilki - x fe+ i|| 2 . (6.35) 
fc=i 

This is a special instance of Problem 16. II with p = m — 1 and, for every i G {1, . . . , to}, /j = and 

fid, if* = l; 

-Id, if * = *; + !; (6.36) 



J ki 



(Wk G {1, . . . , to — 1}) = ^ +1 || • || 2 and 

0, otherwise. 



We can derive from Algorithm 16.21 an algorithm which, by Theorem 16. 3| generates orbits that 
are guaranteed to converge weakly to a solution to (|6.35p . Indeed, in this case, (|6.2p yields /? = 
1/(2(to — 1)). For example, upon setting j n = 7 G ]0, l/(m — 1)[, A n = 0, Aj >n = 0, a^ n = 0, b i n = 0, 
and fi tn = id f° r simplicity, Algorithm 16.21 becomes 

\(Vt G {2,..., TO}) X ijn+ i = P Cl (T^i^l.n + (1 - l^i)Xi^ n ) ■ 

In the particular case when m = 2 and 7 = 1/2, then UJ2 = 1, (|6.35p is equivalent to finding a best 
approximation pair relative to (Ci,C2) [IS], and (|6.37p reduces to 

[x\ tn+X = P Cl {(xx >n + S 2l n)/2) ^ 
1 ^2,n+l = -^2 ((^l,n + a? 2,n)/ 2 ) • 



7 Variational problems over decomposed domains in Sobolev 
spaces 

In this section, we consider a particular case of Problem 16.11 involving Sobolev trace operators in 
coupling terms modeling constraints or transmission conditions at the interfaces of subdomains. 

7.1 Notation and definitions 

We set some notation and recall basic definitions. For details and complements, see [2|[36 l [39 l l53 t l66]. 

We denote by W N the usual A^-dimensional Euclidean space and by | • | its norm, where N > 2. 
Let f2 be a nonempty open bounded subset of M. N with Lipschitz boundary bdryO. The space 
JET 1 (ft) = {x G L 2 (tt) I Dx G (L 2 (tt)) N }, where D denotes the weak gradient, is a Hilbert space 
with scalar product (• | -} H i^ ■ (x,y) 1— > f n xy + f n (Dx) T Dy. We denote by S the surface measure 
on bdryQ |53[ Section 1.1.3]. Now let T be a nonempty open set in bdryfi and let L 2 (T) be the 
space of square 5-integrable functions on T. Endowed with the scalar product 

(• I -)l 2 (t) : ( v ' w ) ^ f r vwdS > C 7 - 1 ) 
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Figure 1: Decomposition of the domain Q. 



L 2 (T) is a Hilbert space. The Sobolev trace operator associated with Q is the unique operator 
T e B(fT 1 (fi) ) L 2 (bdiyfi)) such that (Vs G C 1 ^)) 

Tx — x|bdryS> Endowed with the scalar product 

(•!•): (x,y)~ I (Dx) T Dy, (7.2) 
Jn 

the space H^ r (n) = {xeff 1 (0) | Ti = 0onT} is a Hilbert space \66\ Section 25.10]. Finally, 
for 5- almost every uj G bdryfi, there exists a unit outward normal vector v{uS). 



7.2 Problem formulation and algorithm 



Problem 7.1 Let be a nonempty open bounded subset of R with Lipschitz boundary bdryfi. 
Let (£li)i<i< m be disjoint open subsets of f2 (see Fig. H]) such that the boundaries (bdry Qi)i<i< m 
are Lipschitz, Q = (Jfci an d 

(Vi G {1, ... ,m}) T^j = int bdry n(bdry^j n bdryfi) / 0, (7.3) 

where intbdryn denotes the interior relative to bdryO. For every i G {1, . . . ,m}, set 

(Vj G {i + 1, . . . ,m}) Tjj = T^j = intbdryniCbdryfij nbdryOj), (7.4) 

let 

J(i) = {j6{l,..,m}N{i} | ?ij?0}, (7.5) 
be the set of indices of active interfaces, let T$: -ff 1 (f2j) — > L 2 (bdryOj) be the trace operator, let 

Hi = FoV^O.) = {* G fl^fii) | Ti x = onT M }, (7.6) 

let fi G ro(7ij), and, for every j G •/(*)) let Ty G ]0, +oo[, let i/?y : L 2 (Tj j) -> R be convex and 
Ty-Lipschitz-differentiable, and set Tjj : Hj — > L 2 (Tij): x i— ► (Tjx)|T tj - The problem is to 

m m 

minimize V] /j^) + V] V] ¥>ij(Ty a?i - Tj» Xj), (7.7) 
iieWi,...,i ra eH m * ^— • *— • 

under the assumption that solutions exist. 
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In the above formulation, each function sc$ is defined on a subdomain flj. The potential fi models 
intrinsic properties of X{ while, for every j G J(i), the potential ipij arising in the coupling term 
models the interaction with the jth subdomain as a function of the difference of the Sobolev traces 
of Xi and xj on Tj,-, i.e., of the jump across the interface between fij and O,-. Such variational 
formulations arise in the modeling of transmission problems through thin layers, of Neumann's 
sieve (transmission through a finely perforated surface) , and of cracks in material [4] (5J El [8] . Note 
that, contrary to these approaches, our setting can handle m > 2 domains as well as nonquadratic 
functions ifij. We also observe that, if each ipij: L 2 (Yjj) — > [0, +oo[ and vanishes only at 0, (17. 7h 
can be regarded as a relaxation of some domain decomposition problems, in which one typically 
imposes the "no-jump" conditions Tjj Xi = Tji xj across interfaces pU [56], [63] . More generally, (17.7ft 
can promote various properties of the jump. For instance, if ip^ = d^. . , where is a closed convex 
subset of L 2 (Yjj), the underlying constraint is Tjj x% — TjiXj G Cij. Unilateral conditions 
can be modeled in this fashion. 



Algorithm 7.2 Set 



^ p max m(\\T k i\\ 2 + ||T iA .|| 2 ) ' ^ 



where p is the cardinality of K = {(k, I) | 1 < k < in, I G «/(&)}, and fix e G ]0, min{l, /?}[, (7 n )neN 
in [e, 2(3 — e], and (A n )„, e N in [0, 1 — e]. For every n G N and every £ G {1, . . . , m}, let yj jTl be the 
unique solution in Tii to the problem 



minimize lnfi(y) + \ [ \Dy - Dx i:Tl + j n D(zi, n + bs 
2 J a 



2 

ny | j 



(7.9) 



where z^ n is the unique weak solution in if 1 (J7j) to the Dirichlet-Neumann boundary problem (vi(u) 
is the unit outward normal vector at u G bdryfij) 



Azi iTl = on Jlj 

z»,n = on T;,, 

vjDzi^ n = ^2 on [J T 



(7.10) 



where, for every j G J(z) 



on bdry \ Tj ,-, 



(7.11) 



and set 

■£i,n+l — ^i,n%i,n (1 ^i,n)(2/i,n "I - Ci,n)) (7. 12) 

where, for every i G {1, . . . , m}, the following hold. 

(i) a^o G Hi. 

(ii) (a^rOneN and (fct, n )neN are sequences in TCi such that 



W / |£>a;,n| 2 < +oo and ^ 




(7.13) 



ra<EN V 114 n€N 

(hi) (Ai jn ) neN is a sequence in [0, 1[ such that SneN |Aj,„ - A n | < +oo. 



32 



7.3 Convergence 



Theorem 7.3 Let ((xj in ) nS N)i<i<m be sequences generated by Algorithm \7.ty Then, for every i G 
{1, . . . , to}, (a^rOngN converges weakly in Tii to a point Xi G H«, anc? (cci)i<i< m is a solution to 
ProblemVn\ 



Proof. Given % G {l,...,m} and j G </(«), we first observe that Tjj G 23 (Hi, L (Tjj)). Indeed, 
since the embedding Hi > H l (Qi) is continuous [661 P- 1033] and Tj G 13 (if 1 ^), L 2 (bdry Qj)), 
the operator Tjj : Hi — * L 2 (Tjj) : x h- > (T^lx; is indeed linear and continuous. Let us now show 
that Problem [7j] is a special case of Problem 16. 11 For every (k,l) G IK and every i G {1, . . . , m}, set 



C/ M = L 2 (T k i) and L Wi = < 



T H , if i = k; 

-J lk , if* = Z; (7.14) 
0, otherwise, 



and note that G *B (Tii,Gkl) since (|7.4|) entails L 2 (T k: i) = L 2 (T^ k ). Thus, (|7.7|) can be written 
as 

minimize ^ fi(xi) + ^ ^Pkli ^2 L kH xA , (7.15) 

i=i (JW)eK v i=i 7 

which conforms to (|6.ip . Next, let us show that Algorithm 17.21 is a particular case of Algorithm 16.21 
To this end, let i G {1, . . . , m} and n G N. Since bdryOj = T^j U UjeJfi) ^Ji we deduce from 
[661 Theorem 25.1] that (|7.10p admits a unique weak solution Zi :U G Hi. Accordingly (|7.2p . [661 
Definition 25.31], (TOTD . and ([EI]) yield 

(Vx G Hi) (x | ^ )fl ) = / {Dx) T Dz i;n 

^ Vij^n) dS 
?6J(i) ' 

/ (Ty cZS 



(T<x) 

UjeJ(i) T i,j x j'6J(i) 



x I v ij,n) L2(T id ) 

ieJ(i) 

E T ;, '•'.;•<<)• ( 7 - 16 ) 

Therefore 2j >n = YljeJti) ~^ij v ij,n an d hence (|7.14p and (|7,lip yield 

fc=HeJ(fc) i=i (fc,i)eK ^i=i 7 

On the other hand, it follows from (|7.6|) and (|7.2p that (|7.9|) is equivalent to 

1 2 
minimize 7„/i(y) + - y - (a^n - *y n (Zi,n + b i>n ))\\ , (7.18) 

y£rii I 
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the unique solution of which is 

y i>n = prox 7n/i I x i: n - 7n ( ^ L* kli Vip M i ^ L Mj x 

j,n I "I" 

(7.19) 

Moreover, dOJ is implied by (j?~X4"j) and (fT5]> . Hence, in view of ([7TT2]) and (j7TT3j) . Algorithm is 
a particular case of Algorithm 16.21 Altogether, Theorem 16.31 asserts that, for every i G {1, . . . , to}, 
the sequence (ccij^nsN converges weakly in 7Yj to a point G Wj, where (xj)i<j< m is a solution to 
Problem EU □ 

Example 7.4 Let y G L 2 (Q). With the same notation and hypotheses as in Problem 17.11 let, for 
every i G {1, . . . , to}, 



Hi x h-> - j \Dx\ 2 - j xy and (Vj G J(i)) Vij = d 2 Cip 



(7.20) 



where Cjj is a nonempty closed convex subset of L 2 (Yij). For every i G {1, . . . , to} the solution to 
the problem 

minimize /j (x) (7.21) 

is the weak solution to the Poisson equation with mixed Dirichlet-Neumann conditions |66l Theo- 
rem 25.1] 

—Ax = y on ilj 

x = on T M (7.22) 

^£^ = on Uiej(i) Tij. 
Problem 17.11 couples these Poisson problems by penalizing the violation of the constraints Ty Xj — 

Tji Xj G CV; • 
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